Skip to content

feat(analytics,proxy): routing headers visualization and routing mode query param - #503

Merged
BolotnyaBoss merged 2 commits into
mainfrom
EPMCDME-14083
Sep 25, 2026
Merged

BolotnyaBoss merged 2 commits into
mainfrom
EPMCDME-14083

Conversation

@yanaSelin

@yanaSelin yanaSelin commented Aug 25, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Adds Switchyard/LiteLLM routing awareness across the Claude plugin: the statusline shows what a
router alias actually dispatched to, codemie models list distinguishes router aliases from
concrete deployments, and the analytics report prices and visualizes routing/classifier cost per
turn and per session. Also converts the Claude statusline to a bundled TypeScript artifact and
tightens explicit-model-choice handling across the CLI.

Changes

Statusline

  • Convert the Claude statusline from a loose .mjs script to TypeScript with normal project
    imports, then bundle it into a single esbuild artifact; deploy only the bundled artifact
    (drop shim-file deployment, exclude .ts sources from the Claude plugin asset copy).
  • Add a "routed to" widget: when the configured model alias is a Switchyard virtual router or a
    declared LiteLLM auto-router (isRouterCatalogEntry), show the actual backend model the last
    turn dispatched to instead of just the alias name.
  • Surface classifier (routing LLM) cost and tokens, and a routing-tier indicator, sourced from
    the same transcript-based cost accounting the analytics report uses.

Model list & resolution

  • codemie models list: add a TYPE column (router vs. model) and widen the ID column so
    long free-form router alias names no longer truncate.
  • codemie-codex: accept an explicitly requested --model/env model even when it is not listed
    in the live catalog (router aliases resolve per request and are not always published as
    deployments), gated to an explicit choice via a new CODEMIE_MODEL_SOURCE (cli/env/default)
    marker set by AgentCLI; also recognize gpt-smart-router/gpt-fast-router-style aliases as
    Codex-compatible.
  • Fix subagent-default-tier pinning to skip itself when the session model differs from the only
    provisioned tier, so a router alias assigned to CODEMIE_HAIKU_MODEL/CODEMIE_OPUS_MODEL
    no longer gets silently overridden for subagents declaring model: inherit.
  • Fetch the full model descriptor (fetchCodeMieLlmModels) instead of ID-only, so router entries
    carry their is_router/litellm_router.is_router flag through to both the CLI model list and
    the statusline's routing signal.

Analytics (cost & routing report)

  • Report: per-turn model/routing timeline, a "Routing" KPI section (sessions routed, classifier
    cost), and a routing-tier chart on the session page; classifier cost/tokens roll into session
    and summary totals.
  • New bedrock-pricing.mjs: detects Amazon Bedrock's regional/cross-region-inference pricing
    premium from the raw backend model id and applies it to the rate card.
  • Fix model-name normalization to also strip own Bedrock alias shape
    (bedrock/us.anthropic.claude-sonnet-5, no version suffix), which previously slipped through
    unnormalized and showed up as a raw deployment path next to a clean model name in the same
    session's model breakdown.
  • New background-request-normalizer proxy plugin: force a concrete haiku deployment for Claude
    Code's own background requests (e.g. auto-title generation) instead of deferring to the
    upstream classifier, which sometimes misrouted these tiny calls to the expensive tier — one
    observed case billed a title-gen call on par with the session's most expensive real turn.

Other

  • Fix local-dev SSO API base resolution to not require the code-assistant-api prefix.

Testing

  • npm run typecheck
  • npm run lint
  • Manual testing: generated HTML analytics report against real routed/unrouted sessions;
    verified statusline "routed to" widget and codemie models list TYPE column locally.

Checklist

  • Code follows project standards
  • CI is green (npm run ci)
  • No merge conflicts with main

EPMCDME-14886

EPMCDME-14887

yanaSelin and others added 2 commits September 24, 2026 20:09
… bundling (EPMCDME-14083)

Squashed history of this branch (21 commits) into one for a clean rebase onto main.

- Extract Switchyard/LiteLLM routing headers into a shared domain layer (routing-headers.mjs),
  injected onto response bodies by a new proxy plugin and read back by the analytics cost
  engine and the Claude statusline.
- Add classifier (routing LLM) cost reporting and a Routing KPI section in the analytics report.
- Add Bedrock regional-endpoint pricing detection/premium (bedrock-pricing.mjs).
- Force a concrete haiku deployment for Claude Code's own background requests (title-gen, etc.)
  instead of deferring to the upstream classifier, which sometimes misrouted them to a pricier tier.
- Convert the Claude statusline to TypeScript and bundle it with esbuild instead of deploying
  loose .mjs sources; deploy the bundled artifact only.
- Preserve explicit --model/env model choices during subagent-default-tier resolution; fix the
  stale AgentCLI test this depends on to expect the intended 3-way CODEMIE_MODEL_SOURCE value.
- Resolve local-dev SSO API base without the code-assistant-api prefix.

Refs: EPMCDME-14083
…ckers

Extend the Switchyard/LiteLLM router catalog helpers so a router is judged
by what it actually dispatches to, not by its own family-agnostic alias name:

- sso.http-client.ts: expose router metadata (is_router, router_type, strategy,
  classifier_model, tiers, litellm_router) plus describeRouter/buildModelLabelIndex
  helpers for rendering a router's per-tier target in a picker.
- codex-models.ts: judge a LiteLLM auto-router by its counterfactual_model instead
  of its alias name, and describe routers in the generated Codex catalog.
- claude.models.ts: same counterfactual_model-based check for the /model picker,
  so a GPT-targeting router no longer appears alongside Claude models/routers;
  describe routers instead of the generic "router" label.

EPMCDME-14083

Generated with AI

Co-Authored-By: codemie-ai <[email protected]>
@BolotnyaBoss
BolotnyaBoss merged commit 954f015 into main Sep 25, 2026
5 checks passed
@BolotnyaBoss
BolotnyaBoss deleted the EPMCDME-14083 branch September 25, 2026 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants